HAL 

archives-ouvertes 


Decomposition of Differential Games 

Adriano Festa, Richard Vinter 


► To cite this version: 

Adriano Festa, Richard Vinter. Decomposition of Differential Games. 2014. hal-01122921 


HAL Id: hal-01122921 
https: //hal.inria.fr/hal-01122921 

Submitted on 4 Mar 2015 


HAL is a multi-disciplinary open access 
archive for the deposit and dissemination of sci¬ 
entific research documents, whether they are pub¬ 
lished or not. The documents may come from 
teaching and research institutions in France or 
abroad, or from public or private research centers. 


L ’archive ouverte pluridisciplinaire HAL, est 
destinee au depot et a la diffusion de documents 
scientifiques de niveau recherche, publies ou non, 
emanant des etablissements d’enseignement et de 
recherche franqais ou etrangers, des laboratoires 
publics ou prives. 


arXiv: 1409.4624vl [math.OC] 16 Sep 2014 
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Abstract 

This paper provides a decomposition technique for the purpose of simplifying the solution of certain zero-sum differential 
games. The games considered terminate when the state reaches a target, which can be expressed as the union of a collection 
of target subsets; the decomposition consists of replacing the original target by each of the target subsets. The value of the 
original game is then obtained as the lower envelope of the values of the collection of games resulting from the decomposition, 
which can be much easier to solve than the original game. Criteria are given for the validity of the decomposition. The paper 
includes examples, illustrating the application of the technique to pursuit/evasion games, where the decomposition arises from 
considering the interaction of individual pursuer/evader pairs. 
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1 Introduction 

We propose a decomposition technique to simplify the 
solution of zero-sum differential games that involve two 
players (the a-player and the 6-player), whose actions 
govern the evolution of the state x. The state trajectory 
associated with open loop policies a(.) and 6(.) (‘open 
loop policies’ are defined below), for a specified initial 
state xq, is given by the (absolutely continuous) solution 
of the differential equation 

x{t) = f{x(t),a(t),b(t)), a.e. 

RO) = Xq . 

Here, /(.,.,.) : R ra x M mi x R m2 —> R" is a given func¬ 
tion. Open loop policies a(.) and b(.) of the two players 
take values in specified sets A C R mi and B C R m2 re¬ 
spectively. We write the solution x(t; Xq , a(.), 6(.)). It is 
assumed that hypotheses are imposed on the data en¬ 
suring that a solution exists and it is unique. We also 
specify a closed set T C R" called the ‘target’. The first 
entry time r for x(t; Xq, a(.), 6(.)) is. 

r := sup{t | x{t\ x 0 , a(.),b(.)) £ T} . 


Email addresses: festa@ensta.fr (Adriano Festa), 
r.vinter@imperial.ac.uk ( Richard Vinter). 


Let A and B be the spaces of open loop policies for the 
a-player and 6-player respectively, namely 

A := {a(.) : [0, oo) -A R mi | a(.) meas. and a(t) € A a.e. }, 
B := {a(.) : [0, oo) -A R m2 | 6(.) meas. and bit) € B a.e. } . 

For a(.) € A and 6(.) € B the pay-off is 

J{x 0 ,a(.),b{.)) = 

[ e~ xt l(x(t; 0, xq, a(.), 6(.)), a(t), b(t))dt , 

Jo 

in which A > 0 (the discount factor) is a given number 
and /(.,.,.) : R" x R mi x R m2 -A R (the payoff inte¬ 
grand) is a given function. Here, r is the first entry time 
for x(t; Xq, a(.), 6(.)). 


Following Elliot-Kalton [6], we interpret ‘closed loop’ 
policies for the a-player and 6-player respectively as 

$ := {</> : B -A A | <f> is non-anticipative}, 

’F := {ip : A —> B \ ip is non-anticipative} . 

Here, ‘</>(.) is non-anticipative’ in the first relation means, 
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‘for any t' > 0, and foi(.), & 2 (.) G B, 

bi(t) = b- 2 (t ) a.e. t G [0,t'] => 

<t>Q>i (•))(*) = 0(^2(•))( i ) for a.e. t G [0,t'] . 

‘^(.) is non-anticipative’ in the second defining relation 
is analogously defined. Using these interpretations, we 
define the upper and lower values u(x) and v(x ) of the 
game, for a given starting start x G K", to be 

u(x) = sup inf J 0(.; x, cj>(b (.)),, &(.)), 
beB 

v(x)= inf sup J(x(.’,x,a(.),il>(a(.))) . 
aeA 

Define the real valued functions and G(. 

with domains in K" x 1 x R" -> K 

F(x, u,p) = Xu + inf sup {p • (— f(x, a , b) — l(x, a, 6))} , 

bGB 

G(x, u,p) = Xu + sup inf {p ■ (— f(x, a, b) — l(x, a, 6))}. 

beB °gA 

There is an extensive literature on precise conditions on 
the data, target, etc., under which u(.) coincides with 
v(.), when u(.) can be characterized as the unique con¬ 
tinuous viscosity solution of the HJI (Hamilton Jacobi 
Isaacs) equation: 

F(x, u, Du) = 0 for x G R”\T, 
u(x) =0 for igT, 

and when maximizing closed loop policies for the a- 
player can be obtained from knowledge of u(.). See [1], 
[3], [12] for expository material on these topics, and [2] 
for numerical aspects. 

In this paper, attention focuses on the upper value func- 
ton u(.) and the associated HJI equation (1). We con¬ 
sider situations in which the target T can be repre¬ 
sented as the union of a finite number of closed sets 7 ], 
j = 

T = U ™ =1 T 3 . 

Here, the &-player, responding to the closed loop policy of 
the a-player, has a choice over which component Tj, j = 
1,... to, to exit into, to minimize the payoff. Consider the 
family of ‘reduced’ value functions Uj(.), j = 1,..., m, 
that result when the target T is replaced by the subset 
%. 

Of interest are cases in which the value functions Uj(.), 
j = 1,..., m, for the target subsets are easier to calculate 
than the value function u(.) for the full target T and 
when u(.) can constructed as the lower envelope of the 
Uj(.y s, thus: 

u(x) = min{itj(:r) | j = 1,..., m}. (2) 


The motivation for seeking a decomposition of this na¬ 
ture is as follows. Optimal control problems are special 
cases of differential games in which the constraint set 
A for the a-player is a single point; there is then only 
one possible open loop policy for the a-player, which 
can therefore be effectively ignored. For optimal control 
problems, the decomposition (2) is always valid, since 
replacing T by one particular Tj amounts to a strength¬ 
ening of the problem constraints, and cannot therefore 
reduce the value. So, for any x and any j, u(x) < Uj(x). 
On the other hand, an optimal policy, for the given ini¬ 
tial state x, must result in the state trajectory exiting 
into 71 for some k. But then u(x) > Uj(x). These in¬ 
equalities validate the decomposition (2). 

When the presence of the a-player is restored and we are 
dealing with a true differential game, decomposition is a 
much more complicated issue. There are nontheless in¬ 
teresting cases when the decomposition can be achieved. 
The goal of this paper is to give criteria for decomposi¬ 
tion, and to illustrate their application. 

We shall assume that the value functions involved are 
unique viscosity solutions of the HJI equation with ap¬ 
propriate boundary conditions. This means that check¬ 
ing the validity of the decomposition reduces to answer¬ 
ing the question: when is the lower envelope of a family 
of viscosity solutions to a particular HJI equation also a 
viscosity solution? In Section 2 we give two criteria {(E) 
and ( C )) under which the answer is affirmative. (E) is 
more general, but (C) is often easier to verify. (C) is sat¬ 
isfied, in particular, when F(x,u ,.) is convex. This is a 
well-known fact: the viscosity solution property is pre¬ 
served under the operation of taking the lower envelopes, 
for convex Hamiltonians. Notice that, for optimal con¬ 
trol problems F(x, u ,.) is always convex, so this fact is 
consistent with the earlier observation that, for optimal 
control problems, regarded as special cases of differen¬ 
tial games, the decomposition is possible. However (C) 
is weaker than ‘full’ convexity of F(x,u,.), because it 
requires us to check, for each x G R”\T, the convexity 
inequality only w.r.t. gradient vectors of the minimizing 
Uj(.y s at x. In the examples, this (restricted sense) con¬ 
vexity condition is satisfied while full convexity fails. We 
provide examples from pursuit/evasion games in which 
the decomposition simplifies computations by reducing 
the state dimensionality. 

Some examples of the decomposition, without detailed 
accompanying analysis were presented in [7]. 

2 Properties of the Lower Envelope of a Family 
of Viscosity Solutions 

Take a function F (.,.,.) : R” xRx R™ —> R and consider 
the partial differential equation 

F(x, u(x), Du(x)) =0. (3) 
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and 


Definition 2.1 Take an open subset fl C R" and a func¬ 
tion u(.) : — > R. Then u(.) is a continuous viscosity 
subsolution of (3) on fl if it is continuous and, for each 
x £ O, 

F(x, u(x),p) < 0, \/p £ D + u(x) . (4) 

u(.) is a continuous viscosity supersolution of (3) on Q 
if it is continuous and, for each x € 12, 

F(x,u(x),p) >0, Vp £ D~u(x). (5) 

u(.) is a continuous viscosity solution of (3) on fl if it 
is both a continuous subsolution and supersolution of (3) 
on tt. 

Here, D + u(x ) and D~u(x) denote, respectively, the 
Frechet superdifferential and subdifferential of the con¬ 
tinuous function u(.) defined on an open subset of R n 
containing the point x: 

D + u(x ) := 

/pen": limsup "«-“(*)-?■(»-*) < of , 

l y^x \x-y\ J 


D u{x) := 

I v^ x \x-y | J 

(For the analysis of this paper it is helpful to define con¬ 
tinuous viscosity solutions in terms of one-sided Frechet 
differentials which is equivalent to the standard defi¬ 
nition in terms of gradients of smooth majorizing and 
minoring functions [4]. 


The following proposition gives conditions under which 
the lower envelope of a collection of continuous viscosity 
solutions of (3) is also a continuous viscosity solution, 
expressed in terms of the limiting superdifferential d L (x) 
of the continuous function u(.) at x: 

d L u(x) := {p | 3 sequences pj —>■ p and 

Xi —» x s.t. pi £ D + u{xi) for each ?'} . 

Proposition 2.2 Take a collection of closed sets Tj C 
R", j = 1 ,,m. For each j, let Uj(.) be a scalar valued 
function with domain R n \Tj ■ Define 

/( x) = {j £ {1,... ,to} | Uj (x) = Min Uji (a;)} 

o' 

for each x £ R"\(U"L 1 Tj) 


E = {x £ W n \(U™ =1 Tj) | Cardinality{I(x)} > 1} . 

Take u(.) : R n \(U^h 1 7 j) R to be the lower envelope 
function 

u(x) = Min{uj(x)} . 

0 

(a) : Suppose that Uj(.) is a continuous viscosity supersolu¬ 

tion of (3) on R n \Tj for each j. Then u(.) is a contin¬ 
uous viscosity supersolution of (3) on R"\(U^L 1 Tj). 

(b) : Suppose thatuj(.) is a continuous viscosity subsolution 

of (3) onR n \Tj for each j, that H {.,.,.) is continuous 
and that, for each a;€ S, Uj(.) is Lipschitz continuous 
on a neighbourhood of x. 

Consider the hypotheses: 

(C): for any any set of vectors {pj \ j £ I(x)} such 

thatpj £ d L Uj(x) for each j £ I{x), and any convex 
combination {Aj | j £ I(x)}, 

F(x, u{x), E * jPj) < E X 3 F ( X ’ U 3( X )’Pj) ■ 

jai(x) j&i{x) 

(E): for any x £ E, any set of vectors {pj \ j £ /(x)} such 
thatpj £ d L Uj(x) for each j £ I{x), and any convex 
combination {Aj | j £ I(x)}, 

F(x, u(x), E A jPj) < 0. 

jei(x) 

(i): (E) => ‘u(.) is a continuous viscosity subso¬ 

lution to (3) on R™\(Ujh 1 Tj) 

(ii): If, additionally, Uj{.) is C 1 on a neighborhood of x 
for each j, then 

u(.) is a continuous viscosity subsolution to (3) 
on R n \(U^ =1 Tj) =► ( E). 

(m): (C) =► (E), 

Comments. 

(i) : The proof of the proposition is based on a well- 
known estimate for one-sided differentials to lower en¬ 
velope functions, in terms of the one-sided differentials 
to the constituent functions (the ‘Max Rule’). Such es¬ 
timates are studied in depth in [13]. 

(ii) : The proposition treats separately the preservation 
of the supersolution and subsolution properties of vis¬ 
cosity solutions under the operation of taking the lower 
envelope, because much weaker hypotheses need be im¬ 
posed in connection wth supersolutions. 

(iii) : We give two sufficient conditions for the lower en¬ 
velope of a famility of continuous viscosity solutions also 
to be a continuous viscosity solution, namely (E) and 
(C). ( C) is a more restrictive condition, but it is useful 
because, as illustrated in the following examples, it can 
be easier to verify. 
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(iv): The proposition is an analytical tool for decompos¬ 
ing a differential game (associated with the value func¬ 
tion u(.)) into a collection of simpler problems. The crit¬ 
ical hypothesis in this proposition is ( E ) (or (C)). (C) 
is automatically satisfied when F(x, u, .) is convex. This 
special case of the proposition is well-known [4], How¬ 
ever (C) imposes a convexity type condition on F(x, u, .), 
only with respect to selected vectors in its domain. In 
some cases, examples of which given below, the restricted 
sense convexity hypothesis is satisfied but the full con¬ 
vexity hypothesis is violated; the proposition thereby 
identifies a new class of differential games for which the 
decomposition is possible. 

Proof of Prop. 2.2. 

(a) : Suppose that Uj(.) is a continuous viscosity su- 
persolution of (3) on R n \7} for each j. Take any 
x G K"\(uyk 1 Tj) and p G D~u(x). Then 

u(x') — u{x) > p ■ (x' — x) — o( \x' — x\) , 

for all x' G K"\(U^L 1 T'). (Here, o(.) : K+ -> K+ is 
some function such that lim s j,o o(s)/s —► 0.) Choose any 
j G I{x). We know that Uj(x) = u(x) and Uj(x') > u(x') 

. It follows that, for all x' G K n \(U^k 1 Tj), 

Uj(x') — Uj(x) > p ■ (x 1 — x) — o(\x 1 — x|). 

But then p G D~Uj(x) and, since Uj is a continuous 
viscosity supersolution, we have F(x,Uj(x),p) > 0. It 
follows that F(x,u(x),p) > 0. Since u(.) is continuous, 
we have established that u(.) is a continuous viscosity 
subsolution of (3) on M n \(l>JL 1 Tj). 

(b) (i): Suppose that Uj(.) is a continuous viscosity 
subsolution of (3) on R n \Tj for each j. Take any 
x G K"\(uyk 1 Tj) and p G D + u(x). We must show that 

F(x,u(x),p) <0. (6) 

Suppose first that x ^ E, i.e. I(x) contains a single index 
value j. Then, since the iq(.)’s are continuous, u(x') = 
Uj(x') for all x' in some neighbourhood of a;. It follows 
that p G D + Uj and so F(x, Uj{x),p){= F(x, u(x),p )) < 
0. We have confirmed (6) in this case. 

It may be assumed then that x G £. Now, Uj(.) is 
Lipschitz continuous on a neighbourhood of x for each 
j G I{x). Since p G D + u(x), it is certainly the case that 
p G d L u(x). Using the property that u{x’) coincides with 
maxing (a/) \j G I(x')} for x' in some neighbourhood of 
x, we deduce from the Max Rule for limiting subdiffer¬ 
entials of Lipschitz continuous functions (see, e.g., [16, 
Thm. 5.5.2]) applied to — u(.) the following representa¬ 
tion for p: 

P= 

jei(x) 


for some convex combination {Aj | j G I(x)} and vectors 
Pj G d L Uj{x), j G /( x). But then, by hypothesis (E), 

F(x,u(x),p) = F(x,u(x), ^2 XjPj) <0. 

j£l(x) 

We have confirmed (6) and so (b)(i) is true. 

(b)(ii): Take any igE. Suppose that the Uj’s are con- 
tinously differentiable of a neighbourhood of x and that 
u(.) is a viscosity solution. Take any convex combination 
{Ai} on I(x). Then, for all x' in some neighborhood of x, 

u(x')-u(x)< ^2 Xj(lli(x') — Uj(x)) 
iei(x) 

< ^2 A iVui(x) ■ ( x' — x) + o(\x r — a;|) . 

iei(x) 

This last inequality tells us that J2lgi(x) AiVuj(a;) is a 
limiting superdifferential of u(.) at x. But then, since 
u(.) is a viscosity subsolution, 

F(x, u(x), E A jPj) <0. 

jei(x) 

We have confirmed that ( E ) is true. 

(b)(iii): Take any convex combination {A;} on I(x) and 
vectors pi G d L ui{x) for i G I{x). It follows from the 
definition of the limiting supergradient that, for each i, 
there exist sequences xt x and pt —> pi such that pt G 
D + m(Xj) for * = 1, 2,... But then, for each i G /( x), 

F(x),vP{x l j),p l j) <0, i = 1,2,..., 

since the u,;(.)’s are viscosity subsolutions. It follows that 
J2jei(x) X j F i x j’ uj ( x j)’Pj) < °> * = 1,2,.... Noting 
the continuity of F(., ,.,) and also the Uj{.)' s, we may 
pass to the limit as * —> oo to obtain 

A jF( x ,Uj(x),pj) < 0. 

jei(x) 

Assume (C). Then 

F(x , u{x), E A jPj) < E x j F i x , u j,Pj) 

jGl(x) 3&I(x) 

which is ( E). 

3 Pursuit Evasion Games 

Pursuer/evader games are examples of the game posed 
in the Introduction. There is an extensive literature on 
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such games, going back to Rufus Isaacs’ work in the 
1960’s, and his monograph [10] contains many examples. 
Expository material is to be found in [8], [12]. We note 
also [5], [11], [9], [11], [14], and [15]. But none of these 
references systematically address decomposions of the 
game, each element of which is generated by a target 
subset. Pursuer/evader games is an application area for 
the methods proposed in this paper; they provide exem¬ 
plar problems, both where decomposion is possible, and 
where it is not. 


We consider zero sum differential games which terminate 
when one of the pursuers is sufficently close to one of the 
evaders, where ‘closeness’ is understood in the sense of a 
specified target. The pay-off is the time until the target 
is attained. We analyse a number of examples, involving 
different numbers of pursuers and evaders, and different 
targets. 

The a-player is the collection of m\ evaders, labelled 
1,..., m i, and the 6-player the collection of ?n 2 pursuers, 
labelled m\ + 1,..., mi + m 2 . The states of individual 
pursuers and evaders #i, ..., x mi and x m +i ..., x mi + m2 
are governed by the equations 


A = 0, and may be summarized as: 


I Maximize Minimize f„ r 1 dt 
{bi}eA 0 


xi(t) = f 1 (x 1 (t), ai(t)) 


(P') 


y (^) — fmi-\-rri2 (^mi+m2 7 ^7772 )> 

(ai(f) 

7 • • * ? ^mi 7 ^1 (0 5 • • • 5 ^7772 (t)) 

€ A i x ... x A mi x Pi x ... x P m2 , a.e. 
in which (cn(.),..., a mi (.)) = </>+ (■),•••, b m2 (•)) 
and t is first entry time into T 
(xi(0),..., x mi _j_ m2 (0)) = (xi ,..., x 7ni + rri2 ) 


for some given (x \,..., x mi +m 2 ) € l n x.,.xt n . Here T 
is a given closed subset of R n x ... x R n . The Hamilton- 
Jacobi-Isaacs equation is 


F(x 1 , . . . , X mi _|_ m2 , D Xl u ,..., D Xrrii +m 2 A) 0, (7) 

in which 


dx 1 . . dx mi . .. 

^ /1 (.2-1, (Jl ) , ... , — 

dx. 


m l + l 


dt 

dx rn i_(_m 2 


dt. 


— fmi + 1 (s-toi+I ) ^l)? ••• i 

= /mi+m 2 (^mi+m2J^m2) • 


The variables aq,..., a mi and bi,b m2 are inter¬ 
preted as controls for the evaders and the pursuers, 
respectively, which are subject to the constraints 


a,i € A t , i = 1,..., m 1 , and bi € Bi, i = 1,..., m 2 . 


F(xi,.. 


*^ 7711+7712 7^15 ' * * 7 ^ 7771+7772 / 

777i 7771+7772 

J2 H, (xi,pi) + H l (xi,-Pi)~l. 

7—1 7=777l+l 


Here H l (xi,p t ) := 

sup Pi ■ f(xi, cii) for i = 1,... mi 

ateAi 

sup Pi ■ f(xi, 6j_ mi ) for i = mi + 1,... mi + m 2 . 

— m 1 — m 1 

( 8 ) 


Here, fi{.,.) ■ R™ x R ri —>• R n , i = 1,... ,mi + m 2 
are given functions, and Ai C M ri , l,...,mi and 
Bi C R ri + m i, 1,..., m- 2 , are given subsets. 

We regard ai,...,a mi and bi,...,b m2 as block 
components of a single evader control and pur¬ 
suer control respectively. Take the state to be 1 = 
col {xi. .., x mi+m2 }. The open loop policy spaces for 
evader and pursuer are 

A := {meas. mappings at : [0, 00 ) —> M r % i = 1,..., mi | 
a,i(t) € Ai a.e. for each *}, 

B ■.= {meas. mappings bi : [0, 00 ) —► R ri+m i, i = m 2 | 
bi(t) G Bi a.e. for each i } . 


3.1 A Single Pursuer/Multiple Evaders Game 

Consider first a case of the pursuit/evasion game, written 
(P 1 ), in which m.\ = m > 1, m 2 = 1 and n = 1 (a 
single pursuer/multiple evaders game in ID space). The 
states of the m evaders, labeled 1,..., m and of the one 
pursuer, labeled m+1, are interpreted as the positions of 
the evaders and pursuer. The game terminates when the 
pursuer is first at a distance r from one of the evaders, 
where r > 0 is a given constant. Accordingly, we take 

T = Ti U ... U T m , 


Write $ for the space of non-anticipative mappings </> : 
B —> A. The game fits the formulation Section 1, with 
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in which, for i = 1,..,, m, 

% ■■= {(xi,.. . ,x m+1 ) | |x m+ i - x t \ < r} . 



The Hamilton-Jacobi-Isaacs equation is 

P (aq,..., , F) Xl u, ..., D Xrn + 1 F) 0 , (9) 

in which 


Proof of Prop. 3.1. Note that, for any i, Ui(x i,..., a; TO+ i) 
depends only on the two variables (a;,;,x TO +i). This is 
because the first entry time into 71 only concerns the 
state trajectories associated the rth evader and the 
pursuer (labelled in + 1). 


F (aq,..., £ m _|_i, Pi , • ■ •, Pm+ 1 ) — 


- X] H \ x UPi) + H m+1 (x m +i , —p m +i) - 1 , 


where 


H l (xi,pi) := sup pi - f(xi,a,i)i = l,...m, 

a,i€Ai 

H m+1 (x m+1 ,p m+1 ) = sup Pm+1 • f(x m+ i,bi) . 

6i6Bi 


( 10 ) 


In view of the hypotheses imposed on the ttj(.)’s, the 
fact that u(.) is a viscosity solution of (7) will follow 
from Prop. 2.2, if we can confirm hypothesis (C) of this 
proposition. Take any z = (aq,..., x m+ i) G M. m+1 \T, 
any index set I(z) (of cardinality l > 1) such that the 
values Ui(z), i € I{z), coincide, and any convex combi¬ 
nation {Aj} from I{z). To simplify, assume index values 
have been re-ordered so that I(z ) = {1,...,, Z}. Take 
also p l € R", i = 1 ,..., l such that 

p l := (0, ...,0,p*,0, ...,0 ,j4 +1 ) G d L Ui(z). (11) 


Now take (P/) to be the modification of (P 1 ), when 7) 
replaces T, i = 1,..., to. Let us assume that, for each i, 
the value function Wj(.) for (P- 1 ) is a continuous viscosity 
solution of (7). The following proposition tells us that we 
can construct a viscosity solution to (9) from the «,(.)’s, 
by taking the pointwise infimum. 

Proposition 3.1 Fori = 1, ... m, let Ui(.) be the upper 
value for (P/). Assume 

(a) : Fori = 1,... m, uf.) is a continuous viscosity solution 

of{ 7) on R m+1 \7i. 

(b) : For anyi,j G {1,... ,?n}, i ^ j, and (aq,.. .,x m+1 ) G 

M m+ i\T such thatUi(x\,... ,x m +±) — Uj(x q,..., a: m +i), 
Ui(.) and Uj(.) are Lipschitz continuous on a neigh¬ 
borhood of ( aq,..., x m +i). 

Then 

u{x i,. • ■, a; m +i) . 

min{ui(a;i,. .. .,u„, ;u(aq... .,x m+1 )} 

is a continuous viscosity solution of (7) on (M x ... R)\T. 


(The possibly non-zero components p\ and p' m+1 of p 1 
appear at the Z’th and (m + l)’th locations. We must 
show r/(Ai, • • •, A;) > 0, where 

i i 

7(Ai, • • •, A i) := ^ A iF{z,Pi) - F(z, ^ A^). 

i —1 *=1 

Noting the special structure (11) of the pfs and the fact 
that F[ l (xi,pi ) = 0 when p\ = 0, for each i, we see that 

T](X U ..., Xi) = 

l 

E (//"'' -pUi) - 

i =1 

l l \ 

j7 m+i (x m+1 ,-E A iPUi) - • 

i=l i=l / 

We achieve a further simplification from the fact that 
F[ l (xi ,.) is positively homogeneous, so F[ l {xi, XiP l rn+1 ) = 
XiH l (xi,p l m+1 ). This gives 


Comment. Suppose hypotheses are imposed, ensuring 
that (1): for each i, the FIJI equation for (P/) has a 
continuous viscosity solution on (R n x ...R")\7i with 
a continous extension to Ti, on which set the solution 
vanishes, and (2): the value function (P 1 ) is the unique 
continuous viscosity solution on (R n x ...R™)\T that 
has a continous extension to Ti, on which set the solu¬ 
tion vanishes. The proposition tells us that, under these 
circumstances, the upper value u(.) for (P 1 ) can be cal¬ 
culated as the lower envelope of the continuous viscosiy 
solutions for the (Pf) ’s. (Notice that, since all upper 
values concerned are non-negative, and each Ui(.) is as¬ 
sumed to have a continuous extension to T, on which 
set it vanishes, the lower envelope has a continuous ex¬ 
tension to T, on which set it vanishes.) 


v(Xi,...,X i) = 

l l 

E A j jr n+1 (* r ^ 1 ,-pi l+1 )-ff m+1 (* ro+ i,-E A iPm+i) • 

i—1 i—1 

But then r/(Ai,..., A;) is non-negative, because the term 
F[ m+1 (x m+ i, .), defined by (8), is convex. The proof is 
complete. 

3.2 A Multiple Pursuers/Single Evader Game 

Consider next a case of the pursuit/evader game, writ¬ 
ten (P 2 ), in which mi = 1, ?ri 2 > 1 and n = 2 (sin¬ 
gle pursuer/multiple evaders). The dynamic behavior of 
each player is modelled as a thrust acting on a mass, 
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in ID space, with saturating damping. The state equa¬ 
tions, governing the position and velocity of each player, 
are taken to be, for i = 2,..., m + 1, 


and 

x\ 



x\ 



[+ 2 J —di(* 2 ) + oi J |+(>J [ -di(x\) + bi-x 

Here, 7+) : R —>■ R, z = 1,..., m +1 are given functions 
satisfying 

\di{y) - di{y ')| < k d \y - y '|, di(y) < c d , (12) 

for all y, y' £ R and i = 1,..., m + 1 for some constants 
kd > 0 and c d > 0. The control actions the players are 
required to satisfy 

|a| < a and |6| < /3j for z = 1,... .m. (13) 

for positive constants a, + ,..., /3 m . We assume that 

/3i > a + 2 x c d for i=l,..., m. (14) 

The game terminates when one of the pursuers overtakes 
the evader. Thus, we take the target to be 

T = TiU...ur m , 

in which, for i = 1,..., to, 

7 1 ■= {(a: 1 = (x\,x \),..., 

x m+1 = {x™ + \x™ +1 )\x\>x\} . 

The HJI equation is 

F 2 Or 1 ,..., x m+1 , D x iu,D xm +ru) = 0, (15) 

in which 


F 2 (x\...,x m+ 1 ,p 1 ,...,p m+1 ) = 

m+1 \ m+1 

Y (~Pi x 2 -P2 d {x l 2 )) ) —ax \pl\ + 'Y (A x IpjI) ■ 


Let ( Pf) to be the modification of (P 2 ), when the target 
Ti replaces T, i = 2 ,..., m + 1. 

Proposition 3.2 Let Ui(.) be the upper value for (P 2 ), 
for i = 2, ... m + 1. Assume 


Then 

u(x 1 ,...,x m+1 ) := 

minOmOr 1 ,... x m+1 ), ..., u m+1 ( Xl , ..., x m+1 )} 

is a continuous viscosity solution of (15) on (R 2 ) m+1 \T. 

Comment. When, for each i, the HJI equation for (P 2 ) 
has a continuous viscosity solution Ui(.) on (R 2 ) m+1 \7i 
(with appropriate boundary values) and the value func¬ 
tion u(.) for (P 2 ) is the unique continuous viscosity solu¬ 
tion on (R 2 ) m+1 \T (with appropriate boundary values), 
the proposition describes how the value function for (P 2 ) 
can be obtained, as the pointwise infimum of the Ui(.) ’s. 

Proof. Note that, for i = 2,..., m + 1, u^x 1 ,..., x m+1 ) 
depends only on the two variables {x l ,x l ), since the 
first entry time into 71 only concerns the state trajec¬ 
tories associated with the z’th pursuer and the evader. 
We write iq( x 1 ,x l ), suppressing irrelevant arguments in 
the notation. Note that by assumptions (12) and (14) 
(which tell us that all evaders can accelerate at a faster 
rate than the evader), u^x 1 , x l ) is finite when x\ > x\. 


The left side of the HJI equation P 2 = 0 can be decom¬ 
posed as 




p2 _ p21 _|_ p22 ^ 

(16) 

where P 21 

and P 22 

, evaluated at ((x\,x 2 ),... 

/m+1 m- 

, , Jy 2 

(P 1 -P 2 )>•• 

•, (P? +1 

,p™ +1 )), are: 



m+1 



p 21 = 

- Y^ X% 1 -P2 d {x\)), 

(17) 


and 

m+1 

P 22 = -a X l+l + Y X \p l 2\ ■ ( 18 ) 

i=2 

We shall make use of the following Lemma, whose proof 
appears in the appendix. 

Lemma 3.3 Let a(.) be the open loop strategy a(.) = 
+1 for the evader, and let a(.) be any other open loop 
strategy. Take initial states for the evader z = 
and z’ = ( 2 + z 2 ) such that z x > Zi and z 2 > 22 - Then 

x\{t\ a(.), z') > x\{t\ a(.), z) for all t> 0, (19) 


a continuous viscosity where t —» (x\,xl)(L, a(.),z ) is the state trajectory for the 

evader, under the open loop strategy a(.) and for initial 
(b): Foranyi,j£{2,...,m+l},iy^j,and(x 1 ,...,x m+1 )G statez. 

(R 2 ) m+1 \T such that Uifx 1 ,... ,x m+1 ) = Uj(x 1 ,... ,x m+1 ), 

Ui(.) and Uj(.) are Lipschitz continuous on a neigh- Fix*, and consider (P 2 ). We deduce from the lemma that 
borhood of (x 1 . ,x m+1 ). the optimal closed loop strategy for the a-player (the 


(a): For i = 2,... m + 1, u,:(.) is 
solution of (J) on (R 2 ) m+1 \7{. 
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evader) is </>(6,;(.)) = +1, for arbitrary initial state (a; 1 = 
(x\,x 2 ),x\ = ( x\,x 2 )) such that x\ > x\. Furthermore, 
if the a-player applies this optimal strategy then, for 
any open loop strategy bp.), the effect of increasing the 
x 2 component of the initial state is to increase the first 
interception time. We conclude that 

x\ —>■ Ui((x\,x 2 ), (x\, x 2 )) is monotone increasing 

. ( 20 ) 

for arbitrary ((#}, ( x\,x l 2 )) € R 3 such that x\ > x\. 


Once again, we shall deduce that the lower envelope u{.) 
of the Ui s is a continuous viscosity solution (15) from 
Prop. 2.2, by verifying hypothesis (C). Take any 2 = 
(a; 1 , ..., x m+1 ) £ (R 2 x ... R 2 )\T, any index set I(z) (of 
cardinality l > 1) such that the values upz), i £ I(z) 
coincide, and any convex combination {A;} from I(z). 
We may assume that index values have been re-ordered 
so that I{z) = {2,+ 1}. For i = 2,..., l + 1, take 
any p 1 £ R", such that 

p 1 ■■= p¥), ( 0 , 0 ),..., 

(0,0), (p\\p^), (0,0),... (0,0)) £ d L Ui(z). (21) 

(We have used the fact that u, depends only on (a; 1 = 
(x\, x 2 ), x l = (x\,x 2 )).) The possibly non-zero compo¬ 
nents (p 1 ] 1 ,p l 2 ) and (p\,p 2 ) of p l appear at the first and 
Pth locations. Note that, by (20), 

P 2’ 1 > 0 for i = 2,..., to + 1. (22) 

Verification of hypothesis (C) requires us to show that 
t?(A 2 , ■ • ■, A i+ i) > 0, where 

1 + 2 1 + 2 

?7(A 2 ,..., A;+i) :=^2XiF 21 (z,pi) - F 21 (z,'^2x i p i ) 

i =2 i—2 

1+2 1+2 

+ ^ XiF 22 (z,pi) - F 22 (z, ^2 XzPi) ■ 

i—2 i—2 

Because F 21 (z,.) is linear, we have 

1+2 1+2 

rj{X 2 ,..., A ;+ i) ■.= '^X i F 22 (z,pi) - F 22 (z,'^2x z p i ) 

i—2 i—2 

= Ci + C 2 , 

where 

1 +1 

ci := ^ (-aA i |p 2 ,1 | +ot\Xip\Yj and 

1— 2 

1 +1 l + l 

C2 := A */ 22 (P*) “ / 22 ( z > XlP ^ > 

2— 2 2=2 



Fig. 1. Value function for a one-pursuer one-evader game 



Fig. 2. Optimal trajectories of the agents in the first compo¬ 
nent over time. X denote the point of capture 

in which 

m+1 

f 22 (((plpl),---,(pT +1 ,P2 +1 ) ■■= E& x lf4l- 

i—2 

But ci = 0 since, by (22), the p^’s allhave the same sign. 
Also, c 2 > 0, by convexity of / 22 (.). We have confirmed 
? 7 (A 2 ,..., A;_(_i) > 0, and the proof of the proposition is 
complete. 

For the special case when m = 2, d(x) = x, a = 1 and 
Pi = P 2 = 0.5, (Figure 1) shows computations of the 
value function with respect to the reduced coordinates 
(y 1 >V 2 ) — (*1 — x 2 ,xl — x 2 ) in R 2 . Figure 2 shows an 
example of the evolution of the positions of the players 
over time, with respect to the original coordinates. Cap¬ 
ture occurs at the point marked X, when pursuer P\ 
overtakes the evader, despite starting farther from the 
evader than pursuer P 2 . 

3.3 A Pursuit/Evasion Game With No Decomposition 

We now provide a simple example illustrating that, for 
a multiple pursuers/single evader game, with target a 


























union of target subsets, each associated with the evader 
and just one of the pursuers, may fail to have a de¬ 
composition. In this example, it is possible to derive 
formulae for the value functions involved, and to test 
the conditions for decomposition directly. 

We denote by (P 3 ) the special case of (P) in which mi = 
1, m 2 = 2 and n = 1. 

fi(xi,a) = a, f 2 (x 2 ,bi) = bi and f 3 (x 3 ,b 2 ) = b 2 . 


Proof. Take any z > 0 and let x = (0, z, —z). Then x € 
R 3 \(T 2 U T 3 ). Also, u 2 (x) = u 3 (x), and u 2 (.) and u 3 (.) 
are continuously differentiable at x. From the formulae 
for the value functions we have 

Vu 2 (z) = (—(1 — a) -1 , (1 — a) -1 ,0), and 
Vtt 3 (z) = ((1 - a) -1 ,0, —(1 - aH 1 ). 

Then, for any A £ (0,1), 


The controls actions of the players are constrained as 
follows: 

a £ A := [—a, +a], 

b\ £ B\ := [—1, —F1] and b 2 £ B 2 := [—1,+1], 
for some a £ (0,1). We take the target to be 

T = ?2 U 71, where 


F 3 (\\7 x u 2 (x) + (1 - \)\7 X u 3 (x)) = 


(3 

(1-/3) 


(—A + (l-A))-l = 


A 

(1-/3) 

2A/3 

(W) 


1-A 

+ (W) 

> 0. 


So condition (E) is violated. Then, u(.) cannot be a con- 
tinous viscosity solution, by Prop 2.2, part (b)(iii). 


= {{xi,x 2 ,x 3 ) | xi = x 2 }, T 3 = {(x 1 ,x 2 ,x 3 ) | Xi = x 3 } 

(In this version of the game, two pursuers chase a single 
evader in ID space. The game terminates when either 
pursuer meets the evader.) Denote by (P|) and (P 3 ) the 
modified games in which the target 7~ is replaced by the 
subsets 72 and T 3 respectively. The HJI equation is 

F 3 (D Xl u, D X2 u, D X3 u) = 0, (23) 


in which 


The true value function u(.) for (P 3 ) is expressed in 
terms of the subset: 


V = {(xi,x 2 , x 3 ) £ R 3 | sgn{a’ 2 - x x ) = -sgn{a; 3 - aq} 
and a \x 3 - xi\ < \x 2 - xi\ < ] + Q |x 3 - aq|} . 


1 H - Ot 


1 — a 


It is 


F 3 (pi,P2,p 3 ) =\p 2 \ + \p 3 1 - a\pi\ - 1 • 


Optimal strategies for both games (P 2 ) and (P 3 ) are: 
the evader moves away from the pursuer, and the pur¬ 
suer moves towards the evader, as quickly as possible. 
A simple calculation based on these observations yields 
upper values for (P|) and (P|), namely: 

u 2 (xi,x 2 ,x 3 ) = (1 - a)- l \x 2 - xi \, 
u 3 (x 1 ,x 2 ,x 3 ) = (1 - a) _1 |a; 3 - x 3 \, 


u(x l,X 2 ,X 3 ) 


min{|a :2 - Xi\, \x 3 - aq|} 

for {xi, x 2 , x 3 ) £ R 3 \V 

\{\X2 - Xi\ + \X 3 - Xi\) 

for {xi, x 2 , x 3 ) £ V . 


We see that u(.) coincides with min{-«i(a:), u 2 (x)}, for 
x £ R 3 \V. But 


u{x) < min{rti(a:), ^(a;)}, for x £ V . 


for all x = {x\,x 2 , x 3 ) £ R 3 . Define u(.) : R 3 —> R to be 
u(x ) = mm{ui(x),u 2 (x)} for ieR 3 . 


Proposition 3.4 u(.) is not a continuous viscosity so¬ 
lution for (23) on R 3 \T. 

Since the upper value for (P 3 ) is a viscosity solution on 
R 3 \T, vanishing on T, we may conclude that u(.) is not 
the value function for (P 3 ). 


(The value function is constructed according to the 
heuristic: each of the pursuers always travels at maxi¬ 
mum speed towards the evader, if both pursuers are on 
the same side of the evader, the evader travels at max¬ 
imum speed in the opposite direction until the evader 
is hit. If, on the other hand, the evader is between the 
two pursuers, the evader travels at maximum speed 
away from the closest pursuer until the two pursuers 
are equidistant. The evader then stops until the evader 
is reached. A check is then carried out that the value 
function is a continuous viscosity solution of (7), has 
a continuous extension to 7~ on which it vanishes, and 
which is therefore the upper value of the game.) 
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Appendix: Proof of Lemma 3.3 

Consider first the case z = z' . Fix t > 0. We examine the 
optimal control problem of 

Minimize —y\{t) subject to 

' (yi(s),y 2 (s)) = (y 2 (s),-d(y 2 (s)) + a{s)), a.e. s G [0,1], 

„ a(s) G [—1, +1], a.e. s G [0,1], ( 2 / 1 ( 0 ), y 2 (0)) = (zi,z 2 ). 

(Notice that the controlled differential equation in this prob¬ 
lem is that governing the motion of the evader.) The data 
for the problem satisfy standard hypotheses for the exis¬ 
tence of a minimizer a*(.) on [ 0 , 1 ], with corresponding state 
trajectory y*(.) (see, e.g. [16, Chap. 2]). We can establish, 
by means of a simple contradiction argument, that the non¬ 
smooth Maximum Principle (see [16, Thm. 6.2.3]) applies 
in normal form. We deduce the existence of a costate arc 
p(.) = (pi(.),p 2 (.)) such that pi(.) = + 1 , and p 2 (.) satisfies 
the differential equation and right endpoint boundary con¬ 
dition 

p 2 (s) = +pi(s) - £{s)p 2 (s) for s G [0, it] and p 2 (t) = 0. 

Here, £; (.) is a Lipschitz continuous function satisfying £(s) G 
co 8 l di(y 2 (s)) a.e., in which 8l di is the limiting subdiffer¬ 
ential. The solution p 2 (.) is strictly positive on [0, t). From 
the ‘maximization of the Hamiltonian’ 

a*(s) = arg max{p 2 (s)a | a G [—1, +1]} = +1, 

a*(.) = a(.) on [0,1]. We have shown that, for any t > 0 and 
initial condition z, o(.) = o(.) maximizes yi ( 1 ). This confirms 
(19) when z' = Z. 

We now show that (19) is true also when Z\ = Z\ and z 2 > z 2 . 
In view of the preceding analysis, we can assume that a(.) = 
a(.). Write (yi(.),y 2 (.)) and ( 3/1 (-), 3 / 2 (■)) f° r the solutions to 
the state equation, for initial states z = ( 21 , 22 ) and z' = 
(zi,z 2 ) respectively. Take any time 1 > 0. By assumption 
3 / 2 ( 0 ) > 3 / 2 ( 0 ). So there are two cases to consider 

(a) : 2 ) 2 ( 1 ) > 2 ) 2 ( 1 ) for all 1 > 0. In this case, since 2/1 (0) — 
2 /i( 0 ) > 0 , we have, as required, 

2 /i(i)- 2 /i(i) = ( 2 /i( 0 ) - 2 /i( 0 )) + [ (y 2 (t) — y 2 (t))dt > 0 . 

(b) There exists t' G (0,2] such that 2)2 (2) > 2)2 (1) for t G 
[0,£ / ) and 2 ) 2 ( 2 ') = y 2 (t'). In this case we show as, in the 
previous case, that yi(t') — yi(t') > 0. We deduce from the 
uniquess of solutions to the differential equation 

3 ) 2 ( 1 ) = d(y 2 ) + 1 , 

on [!',£], for fixed initial condition, that 2 / 2 ( 2 ) = 2 / 2 ( 1 ) for 
t G Hence, again, the required relation 

2 /i( 1 ) - 2 /i( 1 ) = ( 2 / 1 ( 1 ') - 3 / 1 ( 1 ')) + f (■ 3 / 2 (t) - 2 h(t))dt > 0 . 
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